Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AUTO] Optimize the device selection logic for auto-plugin #5549

Closed
wants to merge 1 commit into from
Closed

[AUTO] Optimize the device selection logic for auto-plugin #5549

wants to merge 1 commit into from

Conversation

coneypo
Copy link
Contributor

@coneypo coneypo commented May 7, 2021

Details:

  • Optimize the device selection logic for AUTO plugin
  • Only select devices between CPU and GPU, iGPU will be preferred
  • The selection logic will consider device capabilities and network input precision

Tickets:

  • 50743

@coneypo coneypo requested review from a team May 7, 2021 09:07
@coneypo coneypo marked this pull request as draft May 7, 2021 09:09
@openvino-pushbot openvino-pushbot added ExternalIntelPR External contributor from Intel category: inference OpenVINO Runtime library - Inference category: IE Tests OpenVINO Test: plugins and common labels May 7, 2021
@mashoujiang
Copy link
Contributor

mashoujiang commented May 12, 2021

@coneypo I found this is better to rebase to msj/AutoPlugin directly. the limited device feature PR will rebase to this PR.

return name;
}
return {};
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not network precision. It's precision of input layer..

Copy link
Contributor

@mashoujiang mashoujiang May 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well, I remembered Maxim provided some code like:
auto nGraphFunc = network.getFunction();
for (auto &node : nGraphFunc->get_ordered_ops()) {
// auto type0 = node->input_value(0).get_element_type(); //input
auto type1 = node->input_value(1).get_element_type(); //weights
}
so is that expected? should we use typ1 or type0 as the network precision? there are lots of ops, should we use only the first one? thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

int8 network - if we have FakeQuantize operations in the graph
fp16 network - if we have convolution weights stored in fp16
fp32 network - if we have convolution weights stored in fp32

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused, as this PR overlaps heavily with (much larger) #5545, let's focus on 5545, OR change the 5545 to be merged into the branch from THIS requets.
When both requests are trying to merege to the master, we are reviewing the overlapping delta twice

ok, I will change the 5545 to be merged to this branch. the reason I create 5545 and cherry-pick this commit is because I want to test jenkins CI, it seems 5545 CI is OK, I will merge it to this. thanks!

Let's focus on #5545 , which already contains this PR's commit, I will close this PR later.

@myshevts
Copy link
Contributor

I'm a bit confused, as this PR overlaps heavily with (much larger) #5545, let's focus on 5545, OR change the 5545 to be merged into the branch from THIS requets.
When both requests are trying to merege to the master, we are reviewing the overlapping delta twice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: IE Tests OpenVINO Test: plugins and common category: inference OpenVINO Runtime library - Inference ExternalIntelPR External contributor from Intel
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants